Virtual Server Solution In Plant Control System Upgrade At Cementos Argos Cartagena.
By Mark S. Kuhar
When Cementos Argos decided to upgrade the control system at its Cartagena cement plant in Colombia, they started on an unexpected journey. A journey that would end with the implementation of a virtualized high-availability automation system to host a number of FLSmidth’s software products for operation of the complete plant. And all running flawlessly from the start.
Not a Traditional Upgrade
The Cementos Argos Cartagena plant has been operating for more than a decade. During that time, this forward-thinking plant has sought to continually improve performance and increase production. FLSmidth has been a trusted partner in that journey, supplying the original greenfield plant and supporting the plant through the years since.
After 10 years of successful operation, the plant’s ECS/ControlCenter V7 control system was starting to show its age. Cementos Argos was therefore looking for a new solution to secure the digital future of the Cartagena plant, based on the latest control software, ECS/ControlCenter V8.
Enter into the story Hasmukh Patel, technical manager at FLSmidth. For the previous year, Hasmukh and his team had been working to bring the benefits of virtualization – long enjoyed by the IT industry – to the world of operational technology (OT). And to do so in a standardized way that would reduce cost and overcome implementation challenges.
Achieving the (Almost) Impossible
Virtualization had long been thought almost impossible in an OT environment because it can involve a degree of latency – or delay – in the system. While some latency is OK in most IT applications, e.g., with office-based applications, such as email, databases, internet browsers, etc., there is no such luxury or leeway allowed in the OT environment of a manufacturing facility, such as a cement plant.
“In OT, you need the full power of the computer hardware, software and network resources available all the time. Any latency in performance or software errors due to hardware resource constraints are simply not acceptable. When a control room operator pushes a button to start/stop equipment or change setpoints in an industrialized automation environment, you need those control actions delivered instantly, without delay,” Patel explained.
Conventional wisdom did not deter Patel and his team, however. They developed a virtualization solution tailored to the OT environment and optimized ECS/ControlCenter V8 platform for virtualized environments along the way.
Why make this effort? Because Patel and his team believe virtualization holds significant potential to benefit cement plant customers. Benefits that were ultimately persuasive to the forward-thinkers at Cementos Argos, who gave Patel and his team the chance to demonstrate what they could deliver with their futuristic technology solution.
This turned a traditional control system upgrade into something new: the first FLSmidth virtualized automation system, following the new OT optimized standards developed in-house.
What are the Benefits?
Built-in backup for all, not just critical, systems: A virtualized system delivers much greater peace of mind to plant operators. Backup (or redundancy, in technical terms) is built-in for all systems, rather than just those deemed critical. At Cementos Argos, this means that the plant automation team can rest easy, knowing the new system’s three virtual host servers will always deliver and keep the automation applications running 24/7, without manual intervention, even if one of the physical host servers fails.
Easily managing maintenance and security updates: It’s also easier to keep the new system secure. Unlike in traditional set-ups, where security updates must be installed on each server individually, there is one central operation console for the virtualized system. On top of that, the solution has an easy-to-use backup and restore function, adding additional security and ensuring system uptime.
Another potential benefit of the new virtual solution is a “test scenario” for applications during future software upgrades. This allows customers to test software upgrades before deployment for plant operations, helping to minimize the risks involved, as customers can easily revert back to previous application versions, if any issues occur.
“There are a lot of expected savings in the ease of maintenance, including applying security updates and application updates, as well as in the high availability and inherent automatic fail-over redundancy of all applications. Avoiding just one unscheduled stop during production is an important benefit to us. With this robust, high-availability, fault-tolerant solution, we expect uptime of our plant automation systems to be 99.99%,” said Elkin Gustavo Osorio, process coordinator at Cementos Argos.
Building cooperation between IT and OT teams: Another benefit comes from the need for IT and OT teams to work together with the new system to find new synergies and complement each other’s skills. In this project, the Cementos Argos corporate IT department participated in the Factory Acceptance Test, alongside their OT colleagues, at FLSmidth’s office in Bethlehem, Pa.
IT-OT collaboration was also instrumental in implementation, as all automation network switches had to be reconfigured to allow connection of virtual network switches. Other implementation challenges included validation of IP addresses on all nodes to make sure that no duplication or conflicts occurred, and that hardware had to be installed in existing rack cabinets.
Reduced energy consumption: Virtualized solutions require less power to operate and cool the hardware. This fits both with Cementos Argos’ ambitions to reduce carbon dioxide emissions from cement production and the FLSmidth sustainability program, MissionZero, which aims to significantly lower emissions from the cement industry with the target to operate zero-emission cement plants by 2030.
The bottom line: these benefits combine to offer a significantly lower total cost of ownership over a three- to five-year period.
One Project
The success of virtualization at Cartagena is an important step forward – both for the plant but also for the wider cement industry. A moment from testing at the plant is perhaps the best way of capturing the benefits. During the site acceptance test, one server in the three host-server virtual environment was shut down, followed – a deep breath later – by one more server being shut down.
“This was the defining ‘aha’ moment,” concluded Patel. “In this new virtual platform, applications migrate seamlessly between host servers. When we pulled the power cable, applications just moved on to the next available host server, maintaining uptime performance without interruption. While we had redundancy of some critical applications before, this solution puts it on another, higher level, ensuring all applications continue to run flawlessly and thereby making sure the plant is always capable of achieving its production targets and desired performance benchmarks on emissions, quality, optimization, operational costs.”
This was mission accomplished. And a game changer: proof that the benefits of virtualization are now available in OT environments.
What is Virtualization?
According to Microsoft, virtualization “creates a simulated, or virtual, computing environment, as opposed to a physical environment, that often includes computer- generated versions of hardware, operating systems, storage devices, and more. This allows organizations to partition a single physical computer or server into several virtual machines. Each virtual machine can then interact independently and run different operating systems or applications, while sharing the resources of a single host machine. By creating multiple resources from a single computer or server, virtualization improves scalability and workloads, while resulting in the use of fewer overall servers, less energy consumption, and less infrastructure costs and maintenance.”
